AI Lessons
Lesson 12: Applying filters to faces

Purpose: To anchor an image/filter using the landmark points returned by Face Mesh model, to make the image/filter dynamically follow the face, and switch between multiple filters in real time.

No. of Classes

1 - (Time : 1 hour 30 minutes, Laptops/desktops : 10, Students strength : 15 to 20).

Materials Required

Laptop / Desktop with Internet connections / Wi-Fi, and a webcam on every machine.

Curriculum Content

3. Programming Foundations for AI

  • 3.3 Conditionals and program state
  • 3.5 Media assets in code
  • 3.6 User input and interaction

4. Working with Pre-trained AI Models

  • 4.4 Face detection and facial landmark points
  • 4.8 Mapping model output to an interface

7. Designing AI-enabled Systems

  • 7.3 Integrating models into applications
Prior knowledge
  • Browsers
  • Basics of JavaScript - variables, functions and arrays
  • Loading and drawing images on a p5.js canvas
  • Reading facial landmark points from a live webcam feed
Exercises

Exercise (1)


  • Use p5.js and ml5.js to draw virtual accessories such as a nose, glasses and a halo on the detected face, anchoring each one to the correct landmark point.
  • Build an interactive face filter app where the user can switch between different filters in real time.
  • Use the previous lesson's solution code as the starting code for this lesson
  • Click here to download the accessory images for these exercises

Solutions


Teacher's Instruction:
  1. Download the accessory images linked in Exercise 1. Images have to be loaded in preload() before they can be drawn, or nothing will appear on the screen.
  2. Recap the previous lesson: the model returns numbered landmark points, and the numbering is fixed. A filter stays in place because it is drawn at one of those points every time draw() runs, not because it is stuck to the face.
  3. Explain the following to students:
    • Work through the three decisions involved in placing an accessory - which landmark point to anchor it to, how wide to draw it, and how far to shift it so that it sits over the feature rather than beside it.
    • Ask the students to move closer to and further from the camera. The accessory should grow and shrink with the face, which means its size has to be worked out from the landmark points as well, not fixed to one number.
    • The app has to remember which filter is currently chosen. That is program state - one variable holds the choice, and a key press or a click changes it.
    • Draw the accessory after the video, otherwise the webcam picture covers it up.
  4. Discuss some real world applications of the face mesh model.
  5. Points to Ponder:
    • Few years ago models like face mesh was research; today it is a phone app anyone uses without thinking. What is research today that might feel ordinary in ten years?
    • Every filter you have ever used is built on a model trained by a big company. What happens to faces, languages or places that those companies did not think about?